This objective is amenable to minibatching. The variational posterior tracks the true posterior during gradient updates.

Diffusion models have emerged as a powerful class of generative models by learning to iteratively reverse the noising process. Their ability to generate high-quality samples has extended beyond high-dimensional image data to other complex domains such as proteins, where data distributions are typically sparse and unevenly spread. Importantly, the sparsity itself is uneven. Empirically, we observed that while a small fraction of samples lie in dense clusters, the majority occupy regions of varying sparsity across the data space. Existing approaches largely ignore this data-dependent variability. In this work, we introduce a Data-Dependent Smoothing Walk-Jump framework that employs kernel density estimation (KDE) as a preprocessing step to estimate the noise scale $σ$ for each data point, followed by training a score model with these data-dependent $σ$ values. By incorporating local data geometry into the denoising process, our method accounts for the heterogeneous distribution of protein data. Empirical evaluations demonstrate that our approach yields consistent improvements across multiple metrics, highlighting the importance of data-aware sigma prediction for generative modeling in sparse, high-dimensional settings.

The theoretical sections of the paper appear sound, with the Riemannian approaches and their respective invariance properties being properly established. The authors also discuss multiple possible functions that could be applied on the signal powers to obtain the target variable, and prove how using a linear regression model with the Riemannian feature vectors would be optimal for the identity, log and square roots of the signal power. However, they fail to discuss how often these types of scenarios occur in actual MEG/EEG dataset, and also how the performance would deteriorate in case where a different function of the source signals powers is used. The construction of the toy dataset is well thought out to exploit the invariances provided by the Riemannian metrics and demonstrate their performance in the ideal scenario. But as mentioned previously, some additional toy examples that examine the performance of the different models in sub-optimal conditions would also be useful. In addition, it would be interesting to see how the performance of the log-diag model on the toy dataset is affected by the use of supervised spacial filters, or how the geometric distance changes when supervised or unsupervised spacial filters are used.

Originality: - The authors claim this paradigm towards specifying ML models is novel. It is somewhat difficult for me to assess the originality of this work as it's not exactly my area, but I am inclined to agree that their approach seems new and interesting. Quality: - Section 3.3: It's quite generous to call these "key properties" of the model, as really they refer to the results of this particular instantiation of slice-based learning on this toy dataset. It's definitely nice to see that the approach works on a toy dataset, but I would strongly consider reframing this section. The latter two are adequately addressed in the paper and experiments, but the noise aspect was not really addressed.

The paper addresses the problem of predicting future frames in videos from previously seen frames. This task has been gaining a lot on popularity due to it's applications in different areas [8, 22], but also because it aims at modeling the underlying video representations in terms of motion and appearance. The paper proposes a new Decompositional Disentangled Predictive Auto-Encoder (DDPAE), a framework that combines structured probabilistic models and deep networks to automatically (i) decompose the high-dimensional video that we aim to predict into components, and (ii) disentangle each component to have low-dimensional temporal dynamics that are easier to predict. The main novelty is the automatic decomposition of the video into components that are easier to predict, as well as the disentanglement of each component into static appearance and 2D motion. Positive sides: 1. Well written and easy to read.

Simulated events are key ingredients in almost all high-energy physics analyses. However, imperfections in the simulation can lead to sizeable differences between the observed data and simulated events. The effects of such mismodelling on relevant observables must be corrected either effectively via scale factors, with weights or by modifying the distributions of the observables and their correlations. We introduce a correction method that transforms one multidimensional distribution (simulation) into another one (data) using a simple architecture based on a single normalising flow with a boolean condition. We demonstrate the effectiveness of the method on a physics-inspired toy dataset with non-trivial mismodelling of several observables and their correlations.

This crucially differs from existing methods, which only focus on the spurious concept features, risking the loss of vital main-task information. Furthermore, we make the identification of the subspaces systematic by introducing statistical tests that attribute directions in the embedding space to either the main-task or the spurious concept. The method, which we call Joint Subspace Estimation (JSE), is shown to be robust against the strength of the spurious correlation and to outperform existing concept-removal methods for a Toy dataset as well as benchmark datasets for image recognition (Waterbirds, CelebA) and natural language processing (MultiNLI). A high-level overview of the method is given in Figure 1. Figure 1: High-level overview of Joint Subspace Estimation (JSE) for concept removal: the input x is fed through a neural network f(x), from which we can extract the vector representation z. Within the vector representation, two orthogonal subspaces are identified: one related to the spurious concept (the background), and one to the main-task concept (bird type). JSE estimates the subspaces of the two concepts simultaneously to prevent mixing of spurious and main-task features.

Normalising Flows are non-parametric statistical models characterised by their dual capabilities of density estimation and generation. This duality requires an inherently invertible architecture. However, the requirement of invertibility imposes constraints on their expressiveness, necessitating a large number of parameters and innovative architectural designs to achieve good results. Whilst flow-based models predominantly rely on neural-network-based transformations for expressive designs, alternative transformation methods have received limited attention. In this work, we present Ferumal flow, a novel kernelised normalising flow paradigm that integrates kernels into the framework. Our results demonstrate that a kernelised flow can yield competitive or superior results compared to neural network-based flows whilst maintaining parameter efficiency. Kernelised flows excel especially in the low-data regime, enabling flexible non-parametric density estimation in applications with sparse data availability.